Distorted probability operator for dynamic portfolio optimization in times of socio-economic crisis

A robust optimal control of discrete time Markov chains with finite terminal T and bounded costs or wealth using probability distortion is studied. The time inconsistency of these distortion operators and hence its lack of dynamic programming are discussed. Due to that, dynamic versions of these operators are introduced, and its availability for dynamic programming is demonstrated. Based on dynamic programming algorithm, existence of the optimal policy is justified and an application of the theory to portfolio optimization along with a numerical study is also presented.


Introduction
Starting with political and economical crisis in 1990's and recent global Covid-19 pandemic have caused unstable market periods. Most recently, in the global pandemic, stock prices has witnessed sharp drops and peaks (Batabyal and Killinis 2021). This turmoil event is called a black swan. They are said to be unpredictable with mathematical models as being not reflected in historical data (Phillips 2019; Werther 2017). However, for sure monthly expected rate of return for any financial investment in stock is overestimated for a period of data given turbulence will occur (Focard and Fabozzi 2009). Good market returns are biased with overestimation of the probability of positive extremely high returns. To overcome this, using distortion operators is a well known technique to mitigate optimistic expectations. It is used frequently in behavioural finance (see e.g. Kahneman and Tversky 1979, Kahneman and Tversky 1992, Zhou 2010. It has been motivated by empirical studies in behavioural finance and aims to model the human tendency to exaggerate small probabilities of extreme events (Wakker 2010). In particular, it is only natural to consider distortion operators to estimate the risk during global crisis such as the recent pandemic period. One can hinder the error of naively predicting the future returns with a current state of the market that is changing all the time. The distortion operator is shown to be robust to optimistic high returns. Namely, it is more sensitive to the risk of negative returns than the expectation operator by mitigating the chance of positive returns. In particular, the distortion operator is risk-averse. It has a lot in common with value investment strategies that are shown to outperform market return-risk efficiency even in long term. This is to be observed since economic turmoil in 2008 (see Elze 2012, Majewski et al. 2020. In that respect, models with risk aversion has a long history, and they are represented via alternative approaches. One of these approaches is using concave utility functions modelling risk-aversion (see e.g. Chung and Sobel 1987, Fleming and Sheu 1999, Fleming and Sheu 2000, Jaquette 1973, Jaquette 1976) and the references therein), where the utility functions are subject to satisfy some regularity properties. Another approach uses the so-called coherent risk measures introduced in Artzner et al. (1999).
Here, these risk measures are again subject to satisfy some axioms modelling the random outcomes taking risk awareness into consideration. We refer the reader to Artzner et al. (2007); Ruszczynski (2010); Cheridito et al. (2006); Eichhorn and Romisch (2005); Follmer and Penner (2006); Fritelli et al. (2002Fritelli et al. ( , 2005) and the references therein for further details on the risk measures.
On the other hand, although modelling random outcomes representing gains/losses using probability distortion goes back to at least 1970's (Kahneman and Tversky 1979), its axiomatic incorporation into multiperiod settings is still absent in the literature. There are few recent works in this direction. To mention few, (He and Zhou 2011) studies a portfolio optimization problem in continuous time using probability distortion, whereas (Kun et al. 2018) studies a discrete time controlled Markov chain in infinite horizon. In another work, (Ma et al. yyy) assumes monotonicity of the cost/gain functions and presents the results under that assumption. The reason for scarcity of the literature in multiperiod setting lies in the fact that the distortion operator is of limited use in control problems due to not satisfying "Dynamic Programming Principle" (DPP) or "Bellman Optimality Principle". Namely, a sequence of optimization problems with the corresponding optimal controls is called time-consistent, if the optimal strategies obtained when solving the optimal control problem at time s stays optimal when the optimal control problem (OCP) is solved at time t > s (Bjork et al. 2017). OCP's are of vital importance in various fields of operations research In this paper, we introduce a dynamic version of probability distortion that does not suffer from time-inconsistency. DPP can be applied readily in our framework under controlled Markov chains, and additionally DPP gives the existence of the optimal policy.
The rest of the paper is organized as follows. In Sect. 2, we describe probability distortion on random variables in a static one-period case first. Next, we introduce the concept of dynamic probability distortion on stochastic processes in a multi-temporal discrete time setting. In Sect. 3, we introduce the controlled Markov chain framework that we are going to work on. In Sect. 4, we state and solve our optimal control under study, by characterizing both the optimal value and policies as solutions of the dynamic programming equations. In Sect. 5, we illustrate our results on a portfolio optimization problem and conclude the paper.

Probability distortion
In this section, the concept of probability distortion along with the corresponding operator and its properties are introduced. These definitions are further extended to the multi-temporal/dynamic setting.

Probability distortion on random variables
Let ( , F, P) be a probability space and denote by L ∞ + ( , F, P) the set of nonnegative essentially bounded random variables on ( , F).
Remark 2.1 Lemma 2.1 implies that (2.2), respectively (2.3), is an appropriate property of the distortion function w for modelling risk averse behaviour towards random costs, respectively towards random profits.

Dynamic probability distortion on stochastic processes
The main issue occurs when one tries to extend (2.1) to the multi-period setting. In particular, it is not clear what the "conditional version" of distortion operator is. Hence, first the corresponding operator for multi-temporal dynamic setting is constructed. Fix . Let be the sample space with its respective sigma algebra denoted by F. Let F 0 ⊂ F 1 ⊂ . . . F T ⊂ F be a the filtration, and P being the probability measure on such that ( , F, (F t ) t∈T , P) is the stochastic basis. Let ξ = (ξ t ) t∈T be a discrete time non-negative stochastic process that is adapted to the filtration (F t ) t∈T and uniformly bounded that is sup t∈T ess sup(ξ t ) < ∞. We denote in that case ξ ∈ In particular, the "tower property" of expectation operator fails in distortion operators (see Example 2.1 below.). In the context of stochastic optimization, this implies that the optimization problem becomes "time-inconsistent", i.e. the "Dynamic Programming Principle" (DPP) does not hold. On the other hand, for w(x) = x, the distortion operator (2.1) reduces to expectation operator, whereas with the towering property, and DPP holds.
Analogous to Definition 2.1, we define first dynamic distortion mappings on a filtered probability space ( , F, (F t ) t∈T , P) in a multitemporal setting as follows.
• A one-step dynamic distortion mapping t+1|t : , going backwards iteratively, by properties of w and construction of t , uniform boundedness and F s measurability at each s ∈ [t, . . . , T ] are preserved, such that t (ξ T ) maps ξ T to L ∞ + ( , F t , P). Furthermore, by construction s (·) = s ( t (·)) for 0 ≤ s ≤ t ≤ T . In particular, it is a time-consistent operator.

Proof
The proof is a simple modification of Lemma 2.2.
Next, we illustrate the failure of towering property that causes time inconsistency via the following example.
Example 2.1 Let X and Y be two i.i.d. random variables on some probability space where we use Lemma 2.3 i) in the first equality, above. Similarly, we have On the other hand, we have Hence, ρ 1|0 • ρ 2|1 (ξ 2 ) > ρ(ξ 2 ) by strict concavity of w. We further note that the two expressions would be equal to each other, if w(x) = x.

Controlled markov chain framework
In this section, we are going to introduce the necessary background on discretetime controlled Markov processes (a.k.a. Markov decision processes (MDPs) (see e.g. Hernandez-Lerma and Lasserre 1996) that we are going to work, but using now the dynamic probability distortion framework.
We take the control model with the following components: • X t and A t denote the state and action (or control) space, respectively, which are assumed to be Borel spaces, that is, Borel subsets of complete and separable metric spaces with their corresponding Borel σ -algebras B(X t ) and B(A t ). • For each x ∈ X t , let A t (x) ⊂ A t be the set of all admissible controls in the state x. We assume that A t (x) is compact for t ∈ T and denote as the set of feasible state-action pairs. • We define the system function as for all t ∈T with x t ∈ X t and a t ∈ A t , and i.i.d. random variables (η t ) t∈T on a probability space (Y , B(Y ), P η ) with values in Y that are complete separable Borel spaces. We assume that the mapping (s, x, a) → F(s, x, a, y) in (3.2) is continuous on S t × X t × A t for every y ∈ Y at every t ∈T . • Let ⊗ T t=0 X t and for t ∈ T , and be the filtration of increasing σ -algebras. • Let F t be the family of measurable functions and π t ∈ F t with π t : X t → A t for t ∈T . A sequence (π t ) t∈T of functions π t ∈ F t is called an admissible control policy (or simply a policy), and the function π t (·) is called the decision rule or control at time t. We denote by the set of all admissible control policies. • Let r t (x t , a t ) : X t × A t → R + for t ∈T and r T : X T → R + be the nonnegative real-valued reward-per-stage and terminal reward function, respectively.
For (π t ) t∈T ∈ , we write r t (x t , a t ).
• Let π ∈ and x 0 ∈ X 0 be given. Then, there exists a unique probability measure P π on ( , F) such that, given x ∈ X t , a measurable set B t+1 ⊂ X t+1 and (x t , a t ) ∈ K t , for any t ∈T , we have Here, Q t+1 (B t+1 |x t , a t ) is the stochastic kernel (see e.g. Hernandez-Lerma and Lasserre 1996). Namely, for each pair (x t , a t ) ∈ K t , Q t+1 (·|x t , a t ) is a probability measure on X t+1 , and for each B t+1 ∈ B t+1 (X t+1 ), Q t+1 (B t+1 |·, ·) is a measurable function on K t . We remark that at each t ∈ T , the stochastic kernel depends only on (x t , a t ) rather than the whole history (x 0 , a 0 , x 1 , a 1 , . . . , a t , x t ). By (3.2), we have where I B t+1 denotes the indicator function of B t+1 .
Assumption 3.1 • The reward functions r t (x t , a t ) for t ∈T and r T (x T ) are nonnegative, continuous in their arguments and uniformly bounded i.e. 0 ≤ r t (x t , a t ) < ∞ and 0 ≤ r T (x T ) < ∞. • The multi-function (also known as a correspondence or point-to-set function) x → A t (x) is upper semi-continuous (u.s.c.). That is, if {x m } ⊂ X t and {a m } ⊂ A t (x m ) are sequences such that x m →x, and a m →ā, thenā ∈ A t (x) for t ∈T . • For every state x ∈ X t , the admissible action set A t (x) is compact for t ∈T .

Main Result
For every t ∈T , x t ∈ X t and π ∈ , let be the performance evaluation from time t ∈T onwards using the policy π ∈ given the initial condition x ∈ X t . The corresponding optimal (i.e. maximal) value is then A control policy π * = (π * t ) t∈T is said to be optimal if it attains the maximum in (4.1), that is for all x ∈ X t and for t ∈T . (4.2) Thus, the optimal control problem is to find an optimal policy and the associated optimal value (4.2) for all t ∈ T . We now present the main result of the paper.
Theorem 4.1 The optimization problem (4.1) obeys dynamic programming principle and has an optimal policy π * ∈ . Furthermore, V * t (x t ) is continuous in its argument.

Proof of Theorem 4.1
To prove Theorem 4.1, we need the following key lemma.

Lemma 4.1 Let K be defined as in (3.1). Let
Then, for any x t ∈ X t , there exists a B(X t ) measurable mapping π * t : X → A such that and V * : X t → R is continuous.
Proof By Lemma 4.1 in Rieder (1978), there exists B(X t ) measurable mapping π * : a) is lower semi-continuous in x, as well. Hence, V * (·, ·) is continuous in its arguments.

Lemma 4.2 Suppose Assumption 3.1 holds true. Then, supremum is attained at
Proof We will show only the case for t = T − 1. The others follow going backwards iterative down to t = 0. We first show that The second equality follows by boundedness of r T (), w(·) and Lebesgue dominated convergence theorem. The third equality follows by continuity of w(·), the fourth equality follows by continuity of probability measure, and the fifth equality follows by continuity of transition F(·, ·, ·) as in (3.2). Hence, V T −1 (·, ·) is continuous in its arguments. The result follows by Lemma 4.1.
Now, we are ready to prove Theorem 4.1.

Proof of Theorem 4.1 We have
By Lemma (4.2) again, it admits an optimal policy a * T −2 ∈ A T −2 such that Going backwards iterative, we conclude that dynamic programming holds, (4.1) admits an optimal policy π * ∈ attaining supremum that depends only on s t and on x t at each t ∈T . Furthermore, V * t (·) is continuous again by Lemma (4.2). Hence, we conclude the proof.
Based on our main result, the methodology is as follows. Given the distortion operator along with the controlled Markov process, one checks whether the framework with reward and control sets satisfy the requirements in 3.1, Theorem 4.1 reveals that the dynamic programming can be applied to find the optimal controls along with the optimal value function. The next section exemplifies this.

Model
Suppose an investor has a portfolio of n stocks. The prices of n stocks at t ∈ T are denoted by S t (S 1 t , . . . , S n t ).
The price of stock i ∈ [1, 2, , . . . , n] at time t ∈T , denoted by S i t , has dynamics where −1 < r i < 1 is the proportional return rate of price of ith stock S i t . Let P η (·) denote the joint probability mass function of S t for t ∈ T . Let π = (π t ) t∈T be the policy of the investor that stands for the number of shares of n stocks investor is holding at time t ∈T with where π t is B(S t ) measurable. We assume that the investor has a capacity to be in the long or short position. Namely, we take that π t (x) ≤ C for some C > 0, for all x ∈ S t and t ∈T . We denote by the admissible strategies (π t ) t∈T that are B(S t ) measurable and uniformly bounded by C. We take that the market is self-financing in the sense with Y t+1 being the value of the portfolio and S t+1 being the n-dimensional vector as defined in (5.1) at time t + 1 such that denoting is the difference of the total wealth between time t and t − 1 for t ∈ T . Hence, the reward function at t ∈ T reads as Let w(x) = x 2 for x ∈ [0, 1] be the distortion of the probability function such that for a fixed π T −1 given Y T −1 = y T −1 and S T −1 = s T −1 , the performance measure is defined by Here, in the second equality Lemma 2.3 is used. Hence, going backwards iterative, we have at each time t ∈T The methodology is that the application satisfies Theorem 4.1. Indeed, it is immediate to see that the conditions for Assumption 3.1 are satisfied for the reward function and the action sets. Thus, Theorem 4.1 can be applied. In particular, Theorem 4.1 allows the dynamic programming yielding an optimal strategy (π *

Numerical example
Consider discrete time set T = {0, 1, 2} and two stocks Stock A and Stock B. The prices of Stock A and Stock B at t ∈ T are denoted by S A t and S B t respectively. Prices at t = 0 are denoted by S A 0 = S B 0 = 1. Additionally, suppose that an investor can buy one of Stock A, Stock B or portfolio Stock AB with equal shares of both stocks. His initial wealth to be invested equals 1. An investor can switch between Stock A, Stock B, or trade-off between stocks to share equally current wealth into both stocks at each time t = 0, 1, 2. The current state of wealth changes as stock price goes up or down with movement rate denoted by r A and r B , respectively. They are independent random variables satisfying In particular, the expected value of return for the given share is positive Here A 1|0 denotes the value if the investor starts with putting all his wealth into A. B 1|0 and AB 1|0 denote accordingly. By (5.3), optimal strategy is to invest in Stock B for t = 1 and continuing this investment for t = 2. It is depicted in Fig. 2, below. However, return estimated with dynamic probability distortion is 0.912−1 = −0.088. In particular, it is negative.
Moreover, Fig. 1 shows that if one starts with investing in Stock A at t = 0, then the optimal decision is to switch to Stock B at t = 1. In this case the dynamic operator gives value B 0 = 0.864, and return at t = 2 is 0.864 − 1 = −0.136. Similarly, we see at Fig. 3 that if one starts with investing equal shares into Stock A and Stock B at t = 0, then the investor should switch to Stock B, and the distortion operator gives

Conclusion and future work
This paper discusses assessing multi-period risk of an investment using distortion operators. The peculiarity of these operators lies in the fact that they give optimum controls that are not time consistent in a multi-period framework. The dynamic distortion operator introduced in this paper is the way of multi-period aggregation of the distortion operator that satisfies time consistency and hence dynamic programming principle. Furthermore, it is demonstrated how these operators work and apply to the investment problem. The main economic implication of the presented methodology is as follows. If observed probabilities are distorted but observed outcomes of stock return rate are viable, then the dynamic distortion operators produce optimum multi-period investment strategy that is robust to risk and is return effective.
Further research will be to focus on empirical results of investment strategies based on distortion operators. These results should be compared with alternative strategies using the same set of empirical data. For instance, the proposed framework can be compared with the strategies presented in Elze (2012); Majewski et al. (2020), which outperforms market return-risk efficiency. A scope of downside risk measures and models that are alternatives to the presented investing optimization methodology were proposed and applied by Barro and Canestrelli (2014). Also using second order stochastic dominance (SSD) relations (Batabyal and Killinis 2021) and convex risk measures are alternatives (Ruszczynski 2010;Follmer and Penner 2006;Fritelli et al. 2002) to the proposed methodology in this work, whose performance on empirical data is to be compared.

Declarations
Conflict of interest Kerem Ugurlu is the first author of the manuscript. Kerem Ugurlu has been financially supported by Nazarbayev University under the Project SSH2020016 "Robust Methods in Financial Mathematics and Stochastic Control" during the preparation of this manuscript. There is no conflict of interest between the authors of this manuscript.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.